Mateusz #1

mateum97 · 2023-09-08T16:31:40Z

Things that need to be checked of which I'm unsure:

For QIters 3 6 9 IntervalJoin I tried to imitate the new UDFs.getKeyT3() from public class QITER_E1_IntervalJoin instead of using the existing getArtificialKey and I created UDFs.getKeyTX() with X being from 4-9. In all of them I went for return data.f0.getKey(); since in T3 it returned the key of the first KeyDataP, I don't know if this is correct.
For QNOT IntervalJoin I took the insides of the .apply part straight from the Experimental QNOT IntervalJoin despite it using P10 and not P2, I think it should work fine but maybe there is something that I am unaware of that might need an extra look

…mateusz

…teusz

arianeziehn

I think that is not your latest changes
I am missing the Latecny Loggers, and Q8 and Q9
However, I pushed my version update as they my help you to fix your code

arianeziehn · 2023-09-08T17:07:20Z

src/main/java/Q5_NOTPattern.java

+        DataStream<String> result = patternStream.flatSelect(new UDFs.GetResultTuple());
+
+        result//.print();
+              .writeAsText(outputPath, FileSystem.WriteMode.OVERWRITE);


no Latency Logger

arianeziehn · 2023-09-08T17:07:31Z

src/main/java/Q5_NOTPattern.java

@@ -0,0 +1,104 @@
+package Q_SubmissionSigmodVLDB;


no package required

arianeziehn · 2023-09-08T17:07:38Z

src/main/java/Q5_NOTQuery.java

@@ -0,0 +1,139 @@
+package Q_SubmissionSigmodVLDB;


arianeziehn · 2023-09-08T17:08:18Z

src/main/java/Q5_NOTQuery.java

+        result //.print();
+                .writeAsText(outputPath, FileSystem.WriteMode.OVERWRITE);
+


arianeziehn · 2023-09-08T17:20:45Z

src/main/java/Q5_NOTQuery.java

+                .window(SlidingEventTimeWindows.of(Time.minutes(windowSize), Time.minutes(1)))
+                .apply(new FlatJoinFunction<Tuple3<KeyedDataPointGeneral, Long, Integer>, Tuple2<KeyedDataPointGeneral, Integer>, Tuple2<KeyedDataPointGeneral, KeyedDataPointGeneral>>() {
+                    @Override
+                    public void join(Tuple3<KeyedDataPointGeneral, Long, Integer> d1, Tuple2<KeyedDataPointGeneral, Integer> d2, Collector<Tuple2<KeyedDataPointGeneral, KeyedDataPointGeneral>> collector) throws Exception {


…teusz

arianeziehn

So I checked Q9 and Q8 - for Q9 I took over your solution and made some small changes as I need the classes over night to run on the cluster

I left some comments on the other classes to check

most important if you ever want to merge, you need a git ignore file for our target dir

arianeziehn · 2023-09-11T17:55:42Z

target/classes/Q1_SEQPattern$1.class

please remove target from your pull request

arianeziehn · 2023-09-11T18:25:54Z

src/main/java/Q9_SEQQueryLength2_IntervalJoin.java

+
+        DataStream<Tuple2<KeyedDataPointGeneral, KeyedDataPointGeneral>> result = velStream.keyBy(new UDFs.getArtificalKey())
+                .intervalJoin(quaStream.keyBy(new UDFs.getArtificalKey()))
+                .between(Time.seconds(1), Time.seconds((windowSize * 60) - 1))


I found a better way to do it:
.between(Time.minutes(0), Time.minutes(windowSize))
.lowerBoundExclusive()
.upperBoundExclusive()

So I changed that for all IVJ

arianeziehn · 2023-09-11T20:26:12Z

src/main/java/Q8_SEQQueryLS_IntervalJoin.java

+
+        DataStream<Tuple2<KeyedDataPointGeneral, Integer>> stream = input
+                .assignTimestampsAndWatermarks(new UDFs.ExtractTimestamp(60000))
+                .map(new UDFs.MapKey());


remove that data has a actual key we can use

arianeziehn · 2023-09-11T20:27:03Z

src/main/java/Q8_SEQQueryLS_IntervalJoin.java

+                .intervalJoin(quaStream.keyBy(new UDFs.getArtificalKey()))
+                .between(Time.seconds(1), Time.seconds((windowsize * 60) - 1))


change key assignment

.where(KeyedDataPointGeneral::getKey)
.equalTo(KeyedDataPointGeneral::getKey)

.between(Time.minutes(0), Time.minutes(windowSize))
.lowerBoundExclusive()
.upperBoundExclusive()

…teusz

arianeziehn

Hey, I am a bit confused with the PR right now, can we move now to the master branch as the main branch is now under submission and I do not want to merge anything which is not in the paper to it.

arianeziehn · 2023-09-12T11:49:46Z

src/main/java/Q9_SEQQueryLength2.java

@@ -0,0 +1,92 @@
+import org.apache.flink.api.common.JobExecutionResult;


Q9 is now a duplicate, I added all Q9 but I renamed the classes

arianeziehn · 2023-09-12T18:17:19Z

src/main/java/Q6_ITERQueryI1T3.java

@@ -0,0 +1,113 @@
+import org.apache.flink.api.common.JobExecutionResult;


Q6 is also already there but has other names and I took your Q8 classes, so in essence all should be there now and we can close this PR

arianeziehn · 2023-10-04T09:12:00Z

src/main/java/util/Var10Event.java

+    public Var8Event(){
+        super();
+        this.type = "var8";


I think that we can do a bit smarter by adding the String for type to the constructor of the class, then we just need 1 class to create what we want. I particular, I think we can skip all of them and just add the type to the source function of the type

arianeziehn and others added 8 commits February 20, 2023 17:54

Update README.md

e0dfd6f

init

0fc477f

Merge branch 'master' of https://github.com/arianeziehn/CEP2ASP into …

60f1aa5

…mateusz

Merge branch 'main' of https://github.com/arianeziehn/CEP2ASP into ma…

ee67755

…teusz

pulling changes from main

e57ffdb

Merge branch 'main' of https://github.com/arianeziehn/CEP2ASP into ma…

597ec4e

…teusz

Merge branch 'main' of https://github.com/arianeziehn/CEP2ASP into ma…

2a59b9f

…teusz

coding assignment done

7c0075f

arianeziehn requested changes Sep 8, 2023

View reviewed changes

mateum97 added 6 commits September 8, 2023 23:39

thank you git very nice

0a53371

added Q8 and Q9 logger,hashset,intervaljoin

c01528f

Q5IJ

41085ec

Merge branch 'main' of https://github.com/arianeziehn/CEP2ASP into ma…

2c0b6d0

…teusz

commits from main

5ec64e2

wrong package small fix

c07bfd3

arianeziehn requested changes Sep 11, 2023

View reviewed changes

mateum97 added 7 commits September 12, 2023 00:14

removed target

c2baf94

updated gitignore

a13bcbc

added lower/upper bound exclusive and changed key assignment

276fbff

Merge branch 'main' of https://github.com/arianeziehn/CEP2ASP into ma…

f7843f0

…teusz

Merge branch 'main' of https://github.com/arianeziehn/CEP2ASP into ma…

210156e

…teusz

1v3 input files seqlength 8 10 12 experiment

a137d15

q6iterquery length 12

436456e

arianeziehn requested changes Oct 4, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mateusz #1

Mateusz #1

mateum97 commented Sep 8, 2023

arianeziehn left a comment

arianeziehn Sep 8, 2023

arianeziehn Sep 8, 2023

arianeziehn Sep 8, 2023

arianeziehn Sep 8, 2023

arianeziehn Sep 8, 2023

arianeziehn left a comment

arianeziehn Sep 11, 2023

arianeziehn Sep 11, 2023

arianeziehn Sep 11, 2023

arianeziehn Sep 11, 2023

arianeziehn Sep 11, 2023

arianeziehn left a comment

arianeziehn Sep 12, 2023

arianeziehn Sep 12, 2023

arianeziehn Oct 4, 2023

		result //.print();
		.writeAsText(outputPath, FileSystem.WriteMode.OVERWRITE);

		.intervalJoin(quaStream.keyBy(new UDFs.getArtificalKey()))
		.between(Time.seconds(1), Time.seconds((windowsize * 60) - 1))

		@@ -0,0 +1,92 @@
		import org.apache.flink.api.common.JobExecutionResult;

		@@ -0,0 +1,113 @@
		import org.apache.flink.api.common.JobExecutionResult;

Mateusz #1

Are you sure you want to change the base?

Mateusz #1

Conversation

mateum97 commented Sep 8, 2023

arianeziehn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arianeziehn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

arianeziehn left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment